Network self-organization explains the statistics and dynamics of synaptic connection strengths in cortex

The information processing abilities of neural circuits arise from their synaptic connection patterns. Understanding the laws governing these connectivity patterns is essential for understanding brain function. The overall distribution of synaptic strengths of local excitatory connections in cortex and hippocampus is long-tailed, exhibiting a small number of synaptic connections of very large efficacy. At the same time, new synaptic connections are constantly being created and individual synaptic connection strengths show substantial fluctuations across time. It remains unclear through what mechanisms these properties of neural circuits arise and how they contribute to learning and memory. In this study we show that fundamental characteristics of excitatory synaptic connections in cortex and hippocampus can be explained as a consequence of self-organization in a recurrent network combining spike-timing-dependent plasticity (STDP), structural plasticity and different forms of homeostatic plasticity. In the network, associative synaptic plasticity in the form of STDP induces a rich-get-richer dynamics among synapses, while homeostatic mechanisms induce competition. Under distinctly different initial conditions, the ensuing self-organization produces long-tailed synaptic strength distributions matching experimental findings. We show that this self-organization can take place with a purely additive STDP mechanism and that multiplicative weight dynamics emerge as a consequence of network interactions. The observed patterns of fluctuation of synaptic strengths, including elimination and generation of synaptic connections and long-term persistence of strong connections, are consistent with the dynamics of dendritic spines found in rat hippocampus. Beyond this, the model predicts an approximately power-law scaling of the lifetimes of newly established synaptic connection strengths during development. Our results suggest that the combined action of multiple forms of neuronal plasticity plays an essential role in the formation and maintenance of cortical circuits

Network Self-organization explains the distribution of synaptic efficacies in neocortex

The distribution of synaptic efficacies in neocortex has an approximately lognormal shape. Many weak synaptic connections coexist with few very strong connections such that only 20% of synapses contribute 50% of total synaptic strength. Furthermore, recent evidence shows that weak connections fluctuate strongly while the few strong connections are relatively stable, suggesting them as a physiological basis for long-lasting memories. It remains unclear, however, through what mechanisms these properties of cortical networks arise. Here we show that lognormal-like synaptic weight distributions and the characteristic pattern of synapse stability can be parsimoniously explained as a consequence of network selforganization. We simulated a simple self-organizing recurrent neural network model (SORN) composed of binary threshold units. The network receives no external input or noise but self-organizes its connectivity structure solely through different forms of plasticity. Across a wide range of parameters, the network produces lognormal-like synaptic weight distributions and faithfully reproduces experimental data on synapse stability as a function of synaptic efficacy. Overall, our results suggest that the fundamental structural and dynamic properties of cortical networks arise from the self-organizing forces induced by different forms of plasticity

Striatal Network Models of Huntington's Disease Dysfunction Phenotypes

We present a network model of striatum, which generates “winnerless” dynamics typical for a network of sparse, unidirectionally connected inhibitory units. We observe that these dynamics, while interesting and a good match to normal striatal electrophysiological recordings, are fragile. Specifically, we find that randomly initialized networks often show dynamics more resembling “winner-take-all,” and relate this “unhealthy” model activity to dysfunctional physiological and anatomical phenotypes in the striatum of Huntington's disease animal models. We report plasticity as a potent mechanism to refine randomly initialized networks and create a healthy winnerless dynamic in our model, and we explore perturbations to a healthy network, modeled on changes observed in Huntington's disease, such as neuron cell death and increased bidirectional connectivity. We report the effect of these perturbations on the conversion risk of the network to an unhealthy state. Finally we discuss the relationship between structural and functional phenotypes observed at the level of simulated network dynamics as a promising means to model disease progression in different patient populations

Nonlinear dynamics analysis of a self-organizing recurrent neural network : chaos waning

Self-organization is thought to play an important role in structuring nervous systems. It frequently arises as a consequence of plasticity mechanisms in neural networks: connectivity determines network dynamics which in turn feed back on network structure through various forms of plasticity. Recently, self-organizing recurrent neural network models (SORNs) have been shown to learn non-trivial structure in their inputs and to reproduce the experimentally observed statistics and fluctuations of synaptic connection strengths in cortex and hippocampus. However, the dynamics in these networks and how they change with network evolution are still poorly understood. Here we investigate the degree of chaos in SORNs by studying how the networks' self-organization changes their response to small perturbations. We study the effect of perturbations to the excitatory-to-excitatory weight matrix on connection strengths and on unit activities. We find that the network dynamics, characterized by an estimate of the maximum Lyapunov exponent, becomes less chaotic during its self-organization, developing into a regime where only few perturbations become amplified. We also find that due to the mixing of discrete and (quasi-)continuous variables in SORNs, small perturbations to the synaptic weights may become amplified only after a substantial delay, a phenomenon we propose to call deferred chaos

Irregular firing activity in the network around 10000 time step.

<p>A: spike trains of six randomly selected excitatory neurons during 200 time steps. B: example of an ISI distribution and exponential fit of a typical excitatory neuron. C: histogram of CV values of a network's excitatory units. D: correlations between all neurons. Neurons 201–240 are inhibitory. Network activities within the first 3000 steps are discarded to accommodate for a washout of the arbitrary initial state.</p

Deferred chaos.

<p><b>A</b> Histogram of the immediate Lyapunov exponent. The legend shows the starting times of perturbations for each subfigure. <b>B</b> Histogram of the delayed Lyapunov exponent. <b>C</b> Histogram of the combined Lyapunov exponent. <b>D</b> Time course of the immediate and the delayed Lyapunov exponent. <b>E</b> Time course of the combined Lyapunov exponent. <b>F</b> Histogram of the time a weight perturbation needs to affect the neuron activities. The data correspond to <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0086962#pone-0086962-g002" target="_blank">Fig. 2</a>. Errorbars in D and E indicate standard errors of the mean.</p

Relationship of outdegree vs. normalized Euclidean distance at 1,000th time step in Fig. 2 A.

<p>Scatter plot of the perturbed neurons' outdegrees and the corresponding normalized Euclidean distances between the excitatory weight matrices. Coefficients of correlation between the outdegree and the normalized Euclidean distance are not significantly different from zero.</p

Long-term dynamics of the network.

<p>A: fraction of existing excitatory-to-excitatory connections recorded over 5 million time steps. The inset shows an enlargement of the last 1,000 steps. B: synaptic weight distribution recorded at 20,000th time step. C: synaptic weight distribution recorded at 500,000th time step. D: synaptic weights distributions recorded at 3,000,000th (blue dot) and 4,000,000th (red dot) time step. Blue and red curves in B–D are lognormal fits.</p

Different network activities observed with all plasticities and turning off intrinsic plasticity or iSTDP.

<p> denotes the fraction of excitatory neurons firing at time step . Red line is the identity line with . Network activities within the first 3000 steps are dismissed to accommodate for a washout of the arbitrary initial state.</p